C-E-P-S Journal I Vol.l [N“3| Year 2011 


Montenegro in the PISA Study 


Sasa Milic 1 


Montenegro, a country that has been in transition for the last two dec¬ 
ades, is trying intensively to restructure its socioeconomic system and 
reform the main social systems, such as the education system, health 
care, the judicial system, the social welfare system, etc. Numerous stra¬ 
tegic documents have been adopted in the past decade emphasising the 
importance of making the country’s abundant natural resources func¬ 
tional, and of making the utilisation of human resources in the country 
significantly more effective. In order to achieve improvements in one of 
the key areas of Montenegrin development, human resources, a reform 
of the entire education system was launched in the first years of 21 st cen¬ 
tury. The processes of joining the European Union, whose fundamental 
principles are the free movement of people, goods, services and capital, 
have also significantly increased the need to raise the quality of the edu¬ 
cation of Montenegro’s citizens and to improve the competitiveness of 
the Montenegrin workforce in the labour market. However, we believe 
that the results of PISA testing in 2006 and 2009 suggest that Montene¬ 
gro is far from the proclaimed goals of reform in the field of education, 
and that for the coming years and decades considerable attention should 
be devoted to improvement of the education system. PISA tests should 
be understood in a much wider context, not only as a reflection of cur¬ 
ricular reform and standards of verification and assessment of students’ 
knowledge, but rather as a set of guidelines that indicate the direction in 
which to develop and improve the education system, so that society can 
really ‘invest’ in the education of young people. 

It is a very problematic fact that from the time of testing in 2009 until 
April 2011, nobody in Montenegro published any technical or scientific 
analysis of the success, or rather failure, of Montenegrin students in 
PISA testing. We believe that the use of this study should be significant¬ 
ly increased; not for comparing academic achievements with those of 
students from other countries, but primarily for improving educational 
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policy and defining the strategic orientation of the development of the 
education system in Montenegro. Therefore, the absence of analysis im¬ 
plies an absence of certain professional activities focused on training 
teachers and improving the quality of students’ knowledge. 
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Introductory analysis 

Montenegro, as a country that has been in transition for the last two dec¬ 
ades, is trying intensively to restructure its socioeconomic system and reform 
key social systems, such as the education system, the health system, the judicial 
system, the social welfare system, etc. A number of policy documents adopted 
in the past decade have highlighted the importance of improving functionality 
in the country’s natural resources, as well as of making significantly more ef¬ 
ficient use of the country’s human resources. In order to improve one of the key 
elements in the development of Montenegro, human resources, the first years 
of the 21 st century saw the launch of a reform of the entire education system. 
“With the intention of achieving more dynamic socioeconomic development, 
the most important factor is to ensure staff who have adequate knowledge and 
skills. This is an issue of national importance.” (Basis for the Revision of Cur¬ 
ricula, 2003, p. 6). EU integration processes, whose fundamental principles 
are the free movement of people, goods, services and capital, have also sig¬ 
nificantly increased the need to raise the quality of citizens’ education and the 
competitiveness of the Montenegrin workforce in the labour market. However, 
we believe that the results of PISA testing in 2006 and 2009 indicate that Mon¬ 
tenegro is still far from the proclaimed goals of reform in education, and that 
in the coming years and decades we will need to devote considerable attention 
to improving the education system. The PISA test should be understood in a 
wider context; not only as a reflection of curricular reform, scale testing and the 
evaluation of students’ knowledge, but rather as a set of guidelines that show 
us the direction in which to develop and improve the education system, so that 
society can really ‘invest’ in the education of young people. 

The quality of the Montenegrin education system is largely based on well 
educated teaching staff and their enthusiasm and will to invest their maximum 
in the education of children and young people. However, the social crisis of 
the 1980s and 1990s resulted in the disintegration of Yugoslavia, a number of 
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armed conflicts and wars, economic recession accompanied by enormous infla¬ 
tion, and the collapse of society’s value system. At the end of the 20 th century, 
Montenegro was focused on itself, its needs and how to open the way to inde¬ 
pendence. This orientation towards needs led to a better understanding of the 
situation in which the education system existed. Several pedagogical analysts 
in Montenegro, as well as international organisations and relevant educational 
authorities, came up with a similar list of problems in education when analys¬ 
ing the situation in the education system at the beginning of the 21 st century: 1) 
the weakened role of school integration, 2) the lack of school encouragement in 
students’ motivation for learning and continuous intellectual development, 3) 
the lack of linkage between schools and local communities, 4) the lack of peda¬ 
gogical education for parents, 5) the lack of professional competence and moti¬ 
vation of teachers for professional development, 6) the lack of professional au¬ 
tonomy and accountability of schools and teachers, 7) insufficient professional 
management education for directors, 8) the lack of methodological pluralism 
in the educational process, 9) the low quality of knowledge and insufficient 
capacity for full personal, social and work engagement, 10) the inadequacy of 
grading, 11) the lack of an adequate system of monitoring the implementation 
of educational programmes” (Milic, 2010, pp. 230-231). 

In order to overcome these problems of the education system, in 2001, 
Montenegro began an extensive process of education system reform at all lev¬ 
els, through the adoption of the main strategic document, the “Book of Chang¬ 
es.” This document laid the foundation for the transformation of the educa¬ 
tion system, which should respond to the numerous challenges of today, such 
as globalisation, Europeanisation, individualisation and the harmonisation of 
education systems. In addition, the education system should contribute to the 
development of Montenegrin society as an open and democratic society, a so¬ 
ciety based on economic prosperity and a workforce qualified for the needs of 
the modern labour market, and a society that equally respects individual rights 
and needs. All of these strategic goals are inevitably accompanied by the con¬ 
struction of an education system that should provide high quality teaching and 
learning at all levels. In the 2004/05 academic year, Montenegro commenced 
the implementation of reforms in 20 primary schools, and then successively 
included the remaining primary schools, up to total of 161 schools. The imple¬ 
mentation of the reform processes in the secondary segment of the education 
system (grammar schools and secondary vocational schools) started later, in 
the 2005/06 and 2006/07 academic years. “Based on the student-centred teach¬ 
ing approach, the education system designed and implemented must rely on all 
of these principles, especially on the principle of choice according to individual 
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abilities. By applying this principle, each student is given an opportunity to ex¬ 
press his or her full potential. This way young people will be formed as re¬ 
sponsible citizens who contribute with their activities to the democratisation 
of society. In order to enable students to progress according to their abilities 
and to satisfy their special interests, students at all levels of the system (from 
preschool to high school) are given choice, of a subject or part of a subject’s con¬ 
tents, as well as of methods and forms of work, and also different strategies of 
learning. The aforementioned starting point of Montenegrin education reform 
(student-centred teaching), along with respecting the principle of individua¬ 
tion, has imposed the development of curricula based on learning outcomes. 
This means that they contain the objectives that students should achieve, as well 
as standards of required knowledge. In addition to the curriculum, in order 
to respond to the individual needs of children, it was necessary to modify the 
environment in which students spend time, and to intensify the use of different 
methods and forms of work through carefully planned activities for students. In 
addition, students were offered different textbooks, adapted to the new teach¬ 
ing approach and to the different sources of knowledge that students will, dur¬ 
ing primary and secondary education, learn to use” (Popovic, 2010, p. 165). 

Education in Montenegro has always been a focus of interest for both 
the professional and general public, and therefore changes in the education 
system were extensively monitored. Expectations of education system reform 
were high but seldom unrealistic. “The modern knowledge society requires the 
student first to understand and learn how to study throughout life. The ques¬ 
tion is how to make learning more attractive to the student and not to let it 
become torture and an effort, but rather a need and a source of satisfaction” 
(Kotri, 2010, p. 420). However, it seems that the early effects of the reformed 
education system have not given the expected results, and that schools in Mon¬ 
tenegro have failed to significantly raise the level of motivation and interest of 
students for learning and acquiring high-quality and functional knowledge. It 
also appears that the initial enthusiasm of teachers for introducing changes has 
decreased significantly, and that we are now at a turning point in the implemen¬ 
tation of reforms, faced with the more difficult task of re-motivating teachers to 
implement reform solutions and raise the quality of the teaching process. 

Achievements of Montenegrin students 
in the 2009 PISA test 

A basic analysis of the results of testing conducted in 2009 indicates that 
Montenegrin 15-year-olds are significantly below the OECD average, which is 
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about 500 points out of a maximum 600. Montenegrin students’ underachieve¬ 
ment is more significant compared to leading countries: Shanghai-China, 
South Korea, Finland, Hong Kong, China, Singapore, Canada, New Zealand 
and Japan. 



Figure 1: Achievements of Montenegrin students in the 2009 PISA test. 

We believe that the results obtained by students from the best ranked 
countries could largely be the result of education policies that are substantially 
aligned with the statement that “the school, according to the revolutionary 
changes taking place around us, would have better results if dedicated to the 
equally risky, and perhaps equally quixotic, ideal of preparing students for cop¬ 
ing with a changing world in which they will live” (Bruner, 2000, p. 9). 



Figure 2: Results of Montenegrin students in PISA testing in 2006 and 2009. 

Source: OECD studies from 2006 and 2009 (www.oecd.org/document) 
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Although the number of countries participating in PISA testing in 2006 
(57 countries) increased in the evaluation of students’ academic achievement in 
2009 (65 countries), we must conclude, with dissatisfaction, that Montenegrin 
students were ranked significantly worse than in the previous test. Thus, Mon¬ 
tenegro decreased from 51 st to 54 th place in the reading literacy test, and from 
49 th to 54 th place in the mathematics test, while the largest decline in terms of 
achievement was seen in children’s tests of science, where there was a decrease 
from 48 th to 55 th place. 


Fig 13 20/2006-science 
Figl 3 20/2009-science 
Fig 13 9/2006 - mathematics 
Figl.3 9/2009-mathematics 
Figl 14 9/2006-reading 
Figl. 14 9/2009 - reading 
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Figure 3: Summarised results of Montenegrin students in 2009 and 2006 in 
tests of reading literacy (fig.I.14.9), mathematics (fig.I.3.9) and science literacy 
(fig.I.3.20). 

The special significance of the tests is reflected in achievements in read¬ 
ing literacy, due to the fact that reading literacy has a significant direct impact 
on overall student performance, on achievement in other subjects (apart from 
the native language), and on creating reading culture and training students for 
lifelong learning. “In addition, within certain tasks in the reading literacy of 
students they are asked to present their own opinion. Naturally, the question of 
whether schools in Montenegro provide sufficient space for students to express 
their own opinions, irrespective of the subject, with input to critically reflect 
on, express emotional experiences related to some content or text and to evalu¬ 
ate them. It is therefore necessary to pay special attention to reading, but also 
to the development of critical thinking and the child’s speaking; for example, 
by raising adults’ awareness of the importance of reading for the psychosocial 
development of children, by providing better access to books for children, by 
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developing reading habits and motivation for reading, and by ongoing support 
to children in developing skills for reading literacy” (Petricevic, Backovic & 
Ostojic, 2008). In the light of recognition of the importance of reading literacy 
for the overall intellectual development and academic achievement of students, 
the results that Montenegrin students achieved in the 2009 PISA test were very 
discouraging. In terms of the different categories of questions in the reading lit¬ 
eracy test, Montenegrin students achieved the following results (Fig.L2.20/2009 
- access and retrieval of information in a text already read; Fig. 1 .2.23/2009 - the 
integration and interpretation of a text already read; Fig.I.2.26 / 2009 - reflec¬ 
tion and evaluation of a text already read; Fig. 1 .2.32/2009 - the quality of read¬ 
ing in a continuous text; Fig. 1 .2.35/2009 - the quality of reading in a non-con- 
tinuous text): 


Fig I 2 35/2009 
Figl 2 32/2009 
Fig I 2 26/2009 
Fig l 2 23/2009 
Figl 2 20/2009 
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Figure 4: Achievements of Montenegrin students in the reading literacy test. 

According to OECD classification, the achievements of Montenegrin stu¬ 
dents in the reading literacy test are in a category significantly below the OECD 
average, with the achievement of 408 points out of a maximum of 600 points. A 
significant increase in achievements in the 2009 reading literacy test was achieved 
by students from Turkey, Chile and Mexico, all of whom had been ranked be¬ 
hind their Montenegrin peers in previous tests. At the 2009 test, all EU member 
states and some South American countries, such as Uruguay, Trinidad and To¬ 
bago, Colombia and Brazil, achieved better results than Montenegro. The reading 
literacy test showed that enjoyment in reading is an important factor for the im¬ 
proved achievement of Montenegrin students. Accordingly, those who enjoy read¬ 
ing scored the highest average of 457 points, while those who enjoy reading less 
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scored an average of 379 points. The amount of time spent reading also determines 
achievement. Those students who read for 1-2 hours per day achieved an average 
score of 427.8 points, while those who read up to half an hour per day achieved 
an average score of 383 points. Daily online reading also has a positive effect on 
the achievement of Montenegrin students in the reading literacy test. Students in 
the top quarter in terms of time spent reading online achieved an average score of 
416 points, while those from the lower quarter, who read online very little, had an 
average score of 379 points. In terms of the relationship between awareness of the 
importance of links between understanding and remembering information, on 
the one hand, and achievements in the reading literacy test, on the other hand, it 
has been shown that Montenegro is in the category with countries whose students 
have a below average developed awareness of the importance of such strategies 
(compared to the OECD average), which resulted in below average achievements 
in the test (mean index -0.28). Significant disparities appear among Montenegrin 
students in this regard: those from the upper quarter have an average score of 423 
points, while those from lower quarter have an average score of 388 points. There 
is a negative correlation in terms of students’ awareness of the importance of ef¬ 
fective strategies in summarising information and achievements in the reading 
literacy test. In this regard, Montenegro was in the same group as countries such as 
Indonesia, Jordan and Kazakhstan (which is common), and there is a very strong 
negative relationship between below average achievement in the test and a below 
average level of student awareness of information summarising strategies (mean 
index -0.57). It is with regard to these grading criteria that one of the most drastic 
differences among Montenegrin students is evident, with those in the upper quar¬ 
ter achieving an average score of 457 points, while those in the lower quarter have 
an average score of only 378 points. Knowing that 40 points in the test represent 
about the same quantum of knowledge acquired in a school year, we can conclude 
that the underdeveloped strategies of summarising information result in students 
falling behind in the reading literacy test to the extent of more than two school 
years. The inefficiency of the storing and mechanical memorisation of material 
has long been recognised by educational theory and practice, and the achieve¬ 
ments of Montenegrin students in this test confirm this. In fact, it turned out that 
Montenegrin students achieve a negative score of -14.5 points per unit of storage; 
put simply, the more students learn through mechanical memorisation, the worse 
their achievement in the test becomes. A significant departure from the achieve¬ 
ments in relation to the OECD average in the reading literacy test emerges when 
it comes to girls from Montenegro, who are in the large-scale category. Those who 
read tests give us the right to question the objectivity of statements about enjoy¬ 
ment in reading, as, given the large percentage of those who enjoy reading, one 
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would expect accomplishments in the test to be significantly higher. Sexual Gen¬ 
der differentiation in this respect is also expressed in Montenegro, and Montene¬ 
grin boys are significantly closer to the OECD average of 71.5% of those who enjoy 
reading, while girls far exceed the Montenegrin average, with 87.3% of them stating 
that they enjoy reading. Such statements by Montenegrin students give quite an 
unrealistic picture of themselves and their learning process. There are similar dis¬ 
crepancies in terms of links between the socioeconomic status of students and the 
enjoyment of reading. Thus the lower quarter of Montenegrin students according 
to socioeconomic status enjoy reading in 76.5% of cases, while the upper quarter 
by this criterion enjoy reading in 80.9% of cases. The PISA study has shown that 
in Montenegro there is a positive correlation between the socioeconomic status of 
students and income inequality, with this increase in disparity resulting in poorer 
achievement of students, a fact that was shown deeply and broadly. Girls represent 
45.9% in this category (OECD average 49.5%), while Montenegrin boys from this 
category are very close to achieving the OECD average with 38.4% (OECD average 
38.7%). According to the results of the student questionnaire, 79.2% of students in 
Montenegro read for enjoyment, which is significantly above the OECD average 
of 63%. Despite their below average achievement in reading literacy, the percent¬ 
age of variance in reader achievement related to various aspects of family heritage 
(such as level of parental education, cultural heritage and the number of books at 
home, resources for home education, wealth, single parenting, family immigra¬ 
tion status or a combination of these aspects) is significant and could affect an 
almost 20% higher achievement if these aspects were positive. Employment of the 
students’ mothers emerged as an important prerequisite for improved success in 
reading literacy tests, as students whose mothers work achieved significantly bet¬ 
ter test results in reading literacy (428 points). 

An interesting result of the PISA study indicates that the location of 
schools in Montenegro does not have a significant influence on the reading 
literacy of students. The average achievement of students in towns and villages 
with fewer than 3,000 inhabitants was around 390 points, while the average 
achievement in Podgorica, Montenegro’s only city with more than 100,000 in¬ 
habitants, was about 413 points. Although there certainly is a difference, it is 
not as large as is often claimed by the Montenegrin professional community. 
According to the research criteria, Montenegro belongs to a small groups of 
countries, such as Sweden, Norway, Estonia etc., where the range of student 
achievement between villages and towns is relatively small. The biggest gaps 
in the achievement of urban and rural students are in Hungary, Bulgaria, Kyr¬ 
gyzstan and Panama, where the difference is more than 80 points, while the 
OECD average is 40 points in favour of city children. 
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Analysis of the possible failure of Montenegrin students 

A common thesis in pedagogical theory in the countries of Central and 
Eastern Europe is that the system of teacher education in post-socialist coun¬ 
tries was one of the most traditional and conservative systems, and that re¬ 
form was not easy undertaken. In the PISA tests, this was shown to be only 
partly correct. For many decades, teacher education in these countries was ori¬ 
ented only towards lectures, the frontal method of work, preparing students 
for formal and rather rigid assessment of student achievement. The focus was 
on students’ academic achievements to a significantly greater extent than in 
the interactive process of teaching and learning, and the results that students 
of different countries in this group achieve are very high; for example, in the 
reading literacy test of students from Estonia they achieved 13 th place, while 
students from Poland achieved 15 th place. It should also be noted that a signifi¬ 
cant number of countries from this group, including Hungary, Latvia, Slovenia, 
the Czech Republic and Slovakia, are within the range of the OECD average, 
or only slightly below it. In the Montenegrin system of higher education for 
teachers, we have made “the most massive changes regarding higher education 
teaching staff, and the foundation of the study programmes of pedagogy and 
psychology at the Faculty of Philosophy. The education of teaching staff has 
been raised from a high to a higher level of education (VII degree) in order to 
properly respond to the continuously growing needs of early education and 
further professional education for working with preschool children. Of special 
significance in the further development of the education system of Montenegro 
is the foundation of pedagogical and psychological studies, as key areas in the 
higher education of teaching personnel... Significantly fewer changes and less 
innovation have been applied to the higher education of teachers and subject 
teachers (languages, science, history, geography, art), and we can speak more 
of the rearrangement of the old curriculum in the education of teachers, rather 
than of substantive changes and improvement. A particular problem is the lack, 
or complete absence, of the pedagogical-psychological and didactic education 
of expert teaching staff in specialised teaching” (Milic, 2010, p. 233). 

Many reform documents have noted that not enough attention is paid 
to initial motivation in the education of young people, and that they are often 
discouraged; however, we can conclude that the educational process in Monte¬ 
negrin primary schools is still burdened with anachronous organisation, a lack 
of pedagogical stimulation of students and still very present excessive workloads. 
The main disadvantages of the curriculum in the Montenegrin education sys¬ 
tem are: the lack of methodological pluralism in the educational process; the 
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inflexibility of the overloaded curricula; the lack of linkage between knowledge 
and the neglect of multi-disciplinary and interdisciplinary knowledge; the low 
level of knowledge and insufficient capacity for complete personal, social and 
work engagement; the lack of an adequate system of monitoring and implemen¬ 
tation of curricula; and the lack of adequate material and technical conditions 
for the implementation of programme requirements. So far, no extensive scien¬ 
tific research on the effects of the curricular reform has been undertaken, and 
it is therefore difficult to state with any sureness the extent to which the above 
mentioned disadvantages have been removed. However, the results of the PISA 
test still suggest that Montenegrin teachers are still not inclined to accept the 
simple pedagogical strategy that the goal of teaching is not the breadth but the 
depth of the material. “The curriculum of the reformed primary school, com¬ 
pared to the unreformed curricula, set different, modern educational trends 
and developments in the world. Three cycles were introduced - at the end of 
the third, sixth and ninth grades - and the objectives at the end of each cycle 
were defined. Through educational programmes, horizontal and vertical repre¬ 
sentation and connectivity of key areas was provided, sometimes as individual 
curricula and sometimes as thematic units related to the existing curriculum. 
Whenever it was possible, correlation between subjects and alignment and po¬ 
sitioning of intersubject areas was built. The curriculum is designed to leave 
enough space for the required elective courses through which students meet 
their interests in certain fields. In parallel with changing curricula, new text¬ 
books were made in order to follow the new curricular reform and solutions, 
offering, for the first time for some subjects, especially in lower grades, manuals 
for teachers with methodological recommendations for the implementation of 
the programme provided topics in the textbook” (Jacimovic, 2010, p. 64). The 
basis of nine-year primary education is partially an open type of curriculum, 
unlike the closed curriculum that was used in Montenegrin schools before, and 
there has been a transition from teaching to a target curriculum where learning 
outcomes are important, which represents one of the most important innova¬ 
tions of the reformed primary school. Programmes do not contain ‘lessons’, but 
rather topics that are conceived according to defined standards of knowledge 
and skills, to which the teacher must lead students using a more flexible sys¬ 
tem of learning, teaching and access, leaving the teacher room to choose the 
way, methods and assets that will lead students to the defined standard. The 
new reform solutions have changed the role of teachers in the learning process, 
but they have also particularly redefined the role of students; the student is 
transformed from a passive listener to an active participant in the process of ac¬ 
quiring knowledge, a role that the new programmes clearly recognise through 
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the envisaged activities of students. Teachers therefore become more active in 
the preparation of materials for teaching and take on the role of ‘moderators’ 
leading the children to achieve the standards. All of this is planned by the re¬ 
form documents, but it seems that many questions still remain open regarding 
the curriculum. Although the new curriculum recommends vertical and hori¬ 
zontal curriculum correlation, the reality is that the Montenegrin educational 
process is often characterised by the absence of an interdisciplinary approach 
to knowledge and, consequently, the inability of Montenegrin students to com¬ 
bine material from different subject areas. 

Dysfunction and a lack of solidity in the knowledge of Montenegrin stu¬ 
dents is certainly one of the key challenges of the Montenegrin education sys¬ 
tem. We believe that the education system still ‘suffers’ significantly from a lack 
of clear linkage between school knowledge and real life, i.e., the application 
of acquired knowledge in everyday situations. We can completely agree with 
the attitude of the famous German pedagogue, Hartmut von Hentig, who on 
a list of ten deficiencies in reform states that “school reform must go further 
(i.e., to be more radical)... and the school has to become for children a place of 
life where you gain experience relevant to the existence of this world” (Hentig, 
1997). We believe that in the case of curricula in Montenegro this remained a 
major problem after the introduction of the new reform solutions. In fact, PISA 
tests show that the Montenegrin curriculum still pays very little attention to 
the applicative value of knowledge, i.e., the adoption of functional knowledge. 

There is an incapacity for critical thinking, as in the learning process the 
development of Montenegrin students’ problem solving ability is not encour¬ 
aged. This claim is fully supported by surveys conducted among students after 
the PISA test, as well as later impressions of students about their achievements. 
The students themselves assessed that their achievements in the PISA tests were 
unsatisfactory. In their opinion, the results are low because in primary school 
they learned mostly by heart; furthermore, they have not learned to use logic 
and they are generally taught “unnecessary things”, with an emphasis on theory 
rather than practice, especially in the subjects of physics, chemistry and biol¬ 
ogy. The general students’ conclusion about the failure in PISA testing is con¬ 
tained in their attitude: “it turned out that we have been taught incorrectly.” 

The process of training teachers to implement reform solutions in the 
education system took place in parallel with the introduction of changes in 
educational institutions in the 2004/2005 school year. However, despite the fact 
that in the period from 2000 to 2008 some 5800 teachers and 200 directors of 
educational institutions were trained and over 350 seminars were organised, 
we believe that the reasons for the unsatisfactory achievement of Montenegrin 
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students in the PISA test can be sought in the insufficient and inadequate train¬ 
ing of teachers. First of all, we believe that the training period was inadequate, 
with a large number of teachers only undertaking one or two three-day training 
sessions. The competent ministry departed significantly from its own initial de¬ 
termination on the importance of the professional development of teachers. All 
of the relevant studies on the effects of professional development suggest that 
“only the persistent, consistent and long-term application of innovation efforts 
in the education system can lead to its permanent foundation and acceptance 
by the teaching staff and systems in general” (Milic, 2002). Despite these find¬ 
ings, the ministry withdrew the intensive professional training of teachers and 
the entire initial training for the implementation of new, reform solutions was 
reduced to only three days of training. Even today, the ministry has almost 
completely abolished external professional development in the form of semi¬ 
nars, instead introducing so-called internal training at the school level, which 
means interpersonal teacher training in the school and peer learning. The rea¬ 
sons for the lack of preparedness of teachers to improve teaching can be found 
in a significant number of conservative teachers who do not accept change. 
“A significant number of teachers have an evident traditionalist-conservative 
consciousness that is sceptical towards any innovation and change in the ex¬ 
isting situation” (Damjanovic, 2001). The unwillingness and lack of openness 
of the teaching staff regarding the changes is shown by the following research 
findings: “Teachers of elementary school education and learning are treated 
primarily as a phenomenon and the monopoly of childhood and adolescence, 
but not adulthood and life in general” (Todorovic, 2002). Besides the above 
mentioned problems related to the quality of teachers, it is important to em¬ 
phasise the inadequate financial motivation of teachers. Although, as early as 
in 2005, the relevant ministry adopted the “Regulation on Improved Teaching 
Positions,” whose primary purpose was to expedite the improvement of teach¬ 
ing practice in schools and to identify those teachers who invest considerable 
efforts in their own professional development and quality of teaching, the ap¬ 
plication of the rules has not occurred in practice. The ministry justified the 
non-application of this regulation with various pretexts related to the need for 
its further treatment; however, it seems that the main reason was insufficient 
recognition of the importance of motivating teaching staff on the part of the 
ministry, as well as a failure to provide funding to ensure an increase in teach¬ 
ers’ salaries in line with the revised titles (teacher-mentor, teacher-counsellor 
and teacher-researcher). In addition, it should be noted that “the material po¬ 
sition of teachers seriously affects the quality of work and the commitment 
of teachers to introducing innovations in their work. This is reflected in the 
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quality of teaching and learning. Low salaries are a serious obstacle to the im¬ 
plementation of the planned reforms” (Damjanovic, 2001). Although this as¬ 
sessment of the Deputy Minister of Education and Science of Montenegro dates 
back to 2001, the facts indicate that the level of teaching staff salaries is still 
not significantly higher, and that this type of obstacle to the implementation of 
reform solutions is still significant. In addition to the above, it should be noted 
that there is an almost complete lack of training sessions or minimal training of 
teachers in advanced elementary school classrooms and in secondary schools 
for the application of the principles of interactive teaching and the development 
of critical thinking in students. 

As one of the reasons for the unsatisfactory success of Montenegrin 
students, the responsible personnel in the Examination Centre of Montenegro 
also state a lack of motivation for testing. Given that the PISA test is not obliga¬ 
tory in terms of the formal assessment of each student individually, and that 
it has no impact on overall success in school, it was noted that some students 
have an insufficiently serious attitude towards PISA testing and other forms of 
external tests that do not have direct consequences for formal assessment. It 
is particularly important to draw attention to the negative trend in the Mon¬ 
tenegrin education system in terms of priorities. Specifically, the relationship 
of Montenegrin students towards the PISA test clearly indicates that students 
absolute priority is their grade, not their knowledge; accordingly, testing with 
informal effects on individual student achievement and assessment does not 
attract their attention. In view of this possible cause of the poor results of Mon¬ 
tenegrin 15-year-old students in international testing, we believe that it can be 
concluded that there is an absence of national/state consciousness amongst stu¬ 
dents and a low awareness of the need for the quality presentation their own 
country through better achievements in PISA testing. It should be added that 
the promotional campaign about the importance of PISA testing is relatively 
limited. In fact, it seems that, apart from the Examination Centre of Montene¬ 
gro as a referent institution for PISA testing, other institutions of the system, 
especially educational institutions, have done little to promote the importance 
of PISA. Students themselves often state that they were not prepared for testing 
at all; nobody worked with them on preparation for testing seven days prior to 
the realisation of the test. One of the common opinions of participants about 
the reasons for the failure in PISA testing is that this generation of young peo¬ 
ple are only concerned about how to finish school, regardless of the quality of 
knowledge acquired and regardless of what awaits them in the future with such 
superficial knowledge. An observation of the responsible personnel from the 
Examination Centre of Montenegro on the implementation of PISA tests in 
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one high school clearly indicates a very low level of student motivation: it was 
noted that in the school over 50% of the students opened up the tests, looked at 
them, concluded that they were too difficult, and 15 minutes after the test began 
handed the test papers back having filled in almost nothing. The PISA test is not 
the only example of the lack of motivation of students to acquire knowledge; 
the numbers of justified or unjustified absences from school indicate a trend of 
increasing student absence from school, especially from natural sciences and 
mathematics classes. Responsible personnel from the Examination Centre of 
Montenegro attribute the lack of students’ interest in achieving better results 
in the PISA test to their being unaccustomed to the use of standardised tests. 
In fact, it turned out that 37% of Montenegrin schools do not use standardised 
tests of knowledge, while this percentage in countries with better success in 
PISA tests is below 20%. We largely agree with the view of the Austrian educa¬ 
tion theorist, Liessmann, who emphasises that “children whose reading skills 
are weak eloquently lamenting in this way after each PISA test does not im¬ 
prove reading; what is needed is ‘motivation’ and self-regulated learning, and 
in some schools ‘motivation’ has become a school subject. Probably in this class 
you learn how to be motivated for nothing. It is frightening that these concepts 
of practical pedagogical nihilism do not scare anybody” (Liessmann, 2009, 
p. 31). However, we believe that Montenegrin students’ motivation to acquire 
knowledge in various fields is at an unacceptably low level. 

The formalisation of continuous assessment in the regular teaching process 
is undoubtedly one of the major causes of failure in PISA testing. In fact, for 
over a decade there has been a tendency towards extremely superficial and un¬ 
founded assessment of student academic achievement in the education system. 
According to the formal success of Montenegrin students, Montenegro should 
be drastically better ranked in PISA testing, because all generations have a high 
percentage of excellent students, and it is not rare for this percentage to exceed 
50% of the total population. The number of Luca 1 (all excellent grades in pri¬ 
mary education) and Luca 2 (all excellent grades in primary and secondary 
education) student awards handed out indicates that Montenegro should be 
ranked significantly better than it is in international testing. This clearly shows 
very little predictive value in assessing the objectivity of the Montenegrin edu¬ 
cation system. 

Pilot projects of inclusive education in the education system of Montene¬ 
gro have been implemented in 2004 preschools and about 10 primary schools. 
Although the results of the implementation in these educational institutions 
are satisfactory, we believe that the achievement of students with disabilities is 
significantly below average, and this is reflected in the overall achievement of 
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Montenegro’s 15-year-old children in PISA testing. In fact, there are significant 
problems with the dissemination of the idea of inclusive education in the entire 
education system, especially in the advanced grades of primary and secondary 
schools. The reason for this situation lies primarily in the lack of training of 
teachers for accepting children with special needs, and the insufficient training 
of teachers for democratisation in the educational process and for respecting 
children’s rights, as well as the fact that in many situations the relevant ministry 
has not complied with the needs of educational institutions that have children 
with special needs in terms of the adaptation of school buildings, thus limit¬ 
ing the number of classes and educational groups in which there are children 
with special needs. This view is confirmed by findings of the Bureau for Educa¬ 
tion indicating a lack of preparedness of teachers for inclusion. “Although most 
teachers agree that the new curricula is completely adapted to the age and abili¬ 
ties of individual students, nearly one fifth of them believe that they do not pro¬ 
vide enough opportunities for the inclusion of children with special needs in 
education. One of the main causes of this discrepancy can be found in the fact 
that so-called individual educational programmes have not yet become a regu¬ 
lar practice” (“Analysis of teaching goals and...”, 2006). We have also observed a 
negative trend of teachers giving positive grades to students with special needs 
even though their achievements do not warrant such grades, a practice that 
teachers employ in order to avoid being subject to criticism. 

In the Montenegrin education system, the issue of the education of gifted 
children is one of the least examined fields of pedagogical and psychological 
work, and it is a question to which little attention is generally paid in educa¬ 
tional institutions. Therefore, it is no surprise that in PISA testing a very small 
percentage of Montenegro’s 15-year-old students managed to pass level 2, clear¬ 
ly indicating a lack of continuous and systematic work with gifted students. 
We believe that some progress could be made in this area if the responsible 
personnel from the Bureau of Education of Montenegro, as the key institution 
for the creation of curricula, were to implement the planned review of stand¬ 
ards in knowledge and examine whether current standards of knowledge really 
encourage students to develop a higher level of thinking. 

The reform of the education system and new legal solutions in the field 
of education have sought to overcome the problem of the number of students 
per class; the maximum number of students per class is 30, and in exceptional 
cases as high as 32, while the law does not define the lower limit, i.e., the mini¬ 
mum number of students in the classroom. “Almost all of the basic principles 
of the educational reform in our country are the basis for the introduction and 
true respect for the individualisation of the educational process. Applying the 
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principles adopted, starting from the principle of democratisation of the system, 
including increased participation of local communities, citizens, their associa¬ 
tions and parents in all educational change and its direct impact on the perfor¬ 
mance of schools and the educational process, through equal opportunities for 
all, according to which countries must guarantee equal rights to education for 
every individual regardless of gender, social and cultural background, religion, 
national origin, physical and mental constitution, etc. over flexibility, which 
implies that the system as a whole must be flexible, particularly regarding cur¬ 
ricula and their openness, over interculturalisation as a way to prepare young 
people to live in a multicultural society, as well as training for the respect of val¬ 
ues such as tolerance, peace, religious, racial and all other differences, provides 
the basis and creates an obligation for the introduction of individualisation in 
the teaching process” (Popovic, 2010, p. 171). Due to the objective impossibility 
of implementing the regulations and the evident lack of school space, as well 
as the total number of new school buildings, the legal imperative is often not 
respected and the number of students in major urban areas frequently exceeds 
the limit, reaching as many as 33-35 students per class, and sometimes even 
more. Such a large number of students in the class simply prevents quality work 
with all teachers and students, raising questions about the applicability of indi¬ 
vidualisation in the educational process, as one of the basic principles outlined 
by strategic reform documents. 

Final reflections 

The process of reform in the Montenegrin education system was launched 
in 2000/2001, and it should be noted that the main reform documents, such as 
the “Book of Changes” and the “Basis for the Revision of Curricula,” are indisput¬ 
ably good quality strategic documents that are still topical. However, it appears 
that not enough has been done regarding the operational goals and objectives 
outlined in these documents, and that in the live teaching process we can still see 
the old dysfunctional elements of the educational process, with fewer elements 
of modern, interactive teaching being evident. In order to improve the quality of 
students’ knowledge in the Montenegrin education system, and to achieve better 
results in the next PISA tests, the relevant institutions plan to publish a PISA 2009 
National Report with recommendations for education policy makers, which 
should contain analysis of the student questionnaire and the school question¬ 
naire, as well as analysis of student achievement in reading literacy, mathematics 
and natural sciences. Additional training is also planned for teachers working 
with tasks tested in PISA tests, with an emphasis on science and mathematics 
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literacy. This will be supplemented by public campaigns regarding PISA testing, 
in order to raise the awareness and motivation of students, teachers, schools and 
society at large about the importance of this test (Jacimovic, 2010). According 
to representatives of the Bureau for Education, in the future, special attention 
should be focused on setting new goals for the process, insisting on procedural 
knowledge, along with the development of higher levels of abstract and critical 
thinking. Responsible personnel will be monitoring the system of schools, pro¬ 
viding an external evaluation of each school once every four years, while leaving 
room for an internal evaluation at the school in the interim. However, according 
R. Novovic, an adviser from the Bureau for Education, the large time interval in 
external evaluation represents a large space for the unsatisfactory work of teach¬ 
ers and schools in general. Although the educational public emphasises that the 
results of the PISA test can serve educational policymakers and provide good 
indicators for assessing the quality of the education system, of concern is the fact 
that from testing in 2009 until now, April 2011, Montenegro has not published 
any technical or scientific analysis of the success, or rather the failure, of Monte¬ 
negrin students in PISA testing. We believe that the use of this study should be 
significantly increased; not for comparing the academic achievements with stu¬ 
dents from other countries, but primarily for improving educational policy and 
defining strategic orientations for the development of the system in Montenegro. 
The absence of analysis implies the absence of certain professional activities fo¬ 
cused on training teachers and improving the quality of students’ knowledge. 

One of the most important findings of PISA testing in Montenegro that 
should attract the attention of educational policy makers is the achievements of 
students who have attended preschool education. Montenegrin students who 
attended preschool education for between one and three years have average 
test results in reading literacy in the range of 516 to 521 points, while those who 
were at kindergarten for up to one year achieved results ranging from 496 to 505 
points. Both of these results are equal to, or even above, the OECD average. This 
is a very good result, and it is encouraging in terms of the further endeavours of 
Montenegrin students to increase the level of knowledge acquired in the Mon¬ 
tenegrin education system. It certainly provides a clear signal that the relevant 
authorities in Montenegro should do significantly more in the area of children in 
preschool education, as the coverage so far is around 27-28% of the total popula¬ 
tion of children aged from 1 to 6 years. 

We believe that the argument put forward by representatives of relevant 
institutions (the Ministry of Education, the Examination Centre, the Bureau of 
Education) that the PISA tests so far have not included students who attend 
classes in the reformed educational process is difficult to support, and is simply 
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an attempt to justify the low academic achievement of 15-year-old Montenegrin 
students. Given that the implementation of the reform started in the 2004/05 
school year, it should be noted that by now all teachers (class and subject teach¬ 
ers - first to ninth grade) should have been included in the professional devel¬ 
opment of institutions outside schools, as well as in professional development at 
school level. This fact is confirmed by representatives of the Bureau for Educa¬ 
tion, the institution responsible for the professional development of teachers. In 
addition, the argument (or excuse’) given is unacceptable due to the fact that 
in schools a subject teacher usually gives classes in different grades, from the 
fifth to the ninth grade; it therefore seems completely illogical to claim that in 
the fifth or sixth grades of primary school teachers are working to significantly 
improve the quality of development, including critical thinking, the quality of 
interpretation of knowledge, etc., while in the eighth and ninth grades (grades 
that provided the sample of 15-year-old students for PISA 2009) work takes place 
in an old-fashioned and conservative manner, a way that leads to passivity and 
students memorising material, resulting in the significantly low rating of Mon¬ 
tenegro in the PISA test. We also believe that this argument is largely untenable 
due to the fact that school is a living organism that cannot simply be divided by 
a ‘Berlin wall’ into a reformed and unreformed part, and it is very unrealistic to 
claim that in one segment of primary school one can expect significantly bet¬ 
ter results, while in the second segment we have very bad results confirmed by 
PISA testing. The frequent opinion of representatives from relevant institutions 
who emphasise that it is not realistic for Montenegrin teachers to change the 
entrenched traditional mode of teaching ‘overnight’ and start working in a new, 
innovative way, is very serious and worrying. It seems unacceptable to refer to 
the period from 2001 to 2011 as ‘overnight’. This argument further loses weight in 
view of the fact that most teachers now working with Montenegrin 15-year-olds 
have undergone some form of training for the implementation of reform solu¬ 
tions by working with students in lower grades of the third cycle (grades 7-9). 

The systematic and long term view is that the failure of Montenegrin stu¬ 
dents in the PISA test has important implications for all segments of the educa¬ 
tion system, particularly at the secondary and higher education levels. The fact 
that Montenegro has substantially implemented the Bologna Declaration and 
its higher education system has been aligned with the basic ‘Bologna’ education 
principles will not be enough in itself to provide quality in higher education 
staff. It is unrealistic to expect that Montenegrin universities will have high- 
achieving students to enrol in light of the poor results in the PISA tests in 2006 
and 2009. Poorer knowledge gained at lower levels of the education system sim¬ 
ply cannot be compensated for at the university level. This will inevitably lead 
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to a lower level of knowledge of Montenegrin university graduates, and conse¬ 
quently a lower rating of higher education degrees obtained at Montenegrin uni¬ 
versities. This trend will have a direct negative effect on the mobility of students 
and teachers, as well as negative consequences in terms of the employment of 
university educated people in Montenegro. 

The state of reforms in the past, which was a condition that covered the 
education system over a period of several years, is now becoming a permanent 
condition. Due to the fact that “there are three indisputable facts on which edu¬ 
cation must rely in the world today, at the beginning of the 21 st century: the 
growth and strengthening of the multicultural movement, the global perspective 
of world problems and the stunning speed of overall technological development. 
All three of these facts change the nature, structure and objectives of education 
systems in the world” (Herrera & Mandic, 1989). 

The social changes that emerged from the disintegration of former Yu¬ 
goslavia - wars in neighbouring countries, the enormous increase in the refu¬ 
gee population (which at one point was close to 20% of the total population 
of Montenegro), drastic economic decline, etc. - have caused significant chang¬ 
es in the system of social values and the general social devaluation of moral 
norms. Accompanying this, there has been a process of debasement in values of 
knowledge, and the results of the PISA tests, unfortunately, support the idea that 
knowledge is still ranked very low on the scale of social values in Montenegro. 
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